Skip to content

fix(resmgr): a host being deauthorised has not gone away - #47

Merged
PF9-pushkar merged 2 commits into
mainfrom
pushkar/host-404-window
Aug 19, 2026
Merged

fix(resmgr): a host being deauthorised has not gone away#47
PF9-pushkar merged 2 commits into
mainfrom
pushkar/host-404-window

Conversation

@PF9-pushkar

Copy link
Copy Markdown
Collaborator

Two fixes, both patch-level, released as v0.1.9.

A host being deauthorised has not gone away

resmgr answers the per-host endpoints with 404 for minutes after a host's last role is
removed, while GET /resmgr/v2/hosts keeps reporting that host — its roles and its
hostconfig_id — throughout. The host has not gone anywhere.

pcd_host_config_assignment, pcd_host_cluster_role and pcd_host_role believed that 404
and removed live resources from state. The next apply then failed against the reality those
resources never left. Both downstream symptoms were observed on the lab across several runs:

  • 409 HostToHostconfigConflict re-creating an assignment
  • 403 HostInAuthState re-adding a role

hostRecord now cross-checks a 404 against the host list, and only a host absent from both
leaves state. An unreadable list is an error, never an absence — dropping a live resource on an
unverified 404 is the whole thing being prevented. An ordinary read still costs one request;
the extra call happens only on the 404 path.

Applied to all three read paths. If resmgr ever stops 404ing a host it still lists, this
degrades to the fast path and can be retired.

gRPC 1.82.0 -> 1.82.1

Clears the open HIGH dependabot alert (gRPC-Go xDS RBAC and HTTP/2 vulnerabilities). Indirect
dependency; the provider's gRPC server only ever serves the local Terraform CLI over a private
channel and does not use xDS, so real exposure was minimal.

Testing

Static gate green: build, vet, gofmt, golangci-lint 0 issues, all 11 packages' unit tests.

New unit tests cover the window from both sides (404 + still listed = present; 404 + absent
from the list = gone), the fail-closed path when the list cannot be read, and that the ordinary
path stays at one request.

On the evidence, stated plainly: the defect is evidenced from live lab runs (the two
errors above are in recorded logs). The fix ships on unit tests, not on a fresh live
reproduction — reproducing the window requires a deauth, which reliably triggers two open PCD
product bugs and left the lab needing a full rebuild. That trade was not worth another rebuild.

An adversarial audit of this diff raised 12 findings; 11 did not survive verification. The one
that did was a coverage gap: nothing pinned the guard that keeps a per-host failure which is
not a 404 (a 500, an expired token) from being reported as an absence. The shipped code was
correct — all three callers check err before known — but the guard could be deleted with the
suite staying green. Mutation-tested and now covered.

Pushkar Mulay added 2 commits August 18, 2026 17:48
resmgr answers the per-host endpoints with 404 for minutes after a host's last
role is removed, while GET /resmgr/v2/hosts keeps reporting the host, its roles
and its hostconfig_id throughout. Three reads believed that 404 and removed
resources that still existed:

    pcd_host_config_assignment   -> next apply: 409 HostToHostconfigConflict
    pcd_host_cluster_role        -> next apply: 403 HostInAuthState
    pcd_host_role                -> same

Worse than the failed apply, the assignment leaving state is what let a host
config be deleted as unused, which strands the host for good — the case the
delete guard now refuses outright.

So a 404 is no longer taken at face value: hostRecord falls back to the host
list, which is the only view that stays truthful through the window, and a list
that cannot be read is an error rather than an absence. Nothing leaves state on
an unverified 404. The extra request happens on the 404 path alone, so an
ordinary read still costs one call.

Waiting the window out belongs in the writes that must not race it, not in a
Read: a refresh must not block for minutes.

Also bumps the indirect google.golang.org/grpc to 1.82.1 (xDS RBAC and HTTP/2
advisories). The provider's gRPC server only serves the local Terraform CLI over
a private channel and does not use xDS, so exposure was minimal.
@PF9-pushkar
PF9-pushkar merged commit 9c3a978 into main Aug 19, 2026
5 checks passed
@PF9-pushkar
PF9-pushkar deleted the pushkar/host-404-window branch August 19, 2026 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant